network: add DetachedSignatureAvailableCheck#771
network: add DetachedSignatureAvailableCheck#771parona-source wants to merge 3 commits intopkgcore:masterfrom
Conversation
8697f61 to
694a268
Compare
thesamesam
left a comment
There was a problem hiding this comment.
Thanks a lot for this. It works well for me as well.
arthurzam
left a comment
There was a problem hiding this comment.
Some small changes I want, but looks really good.
I even want to run it global wise to know if it finds something
src/pkgcheck/checks/network.py
Outdated
| for url in f.uri: | ||
| for extension in self.detached_signature_extensions: | ||
| yield (f.filename, f"{url}{extension}") | ||
| return [] |
There was a problem hiding this comment.
| return [] |
no need for this return
There was a problem hiding this comment.
This is also done this way in PyPIAttestationAvailableCheck. Should that be changed as well?
I mostly copied that check.
There was a problem hiding this comment.
Applied suggestion. Didn't touch PyPIAttestationAvailableCheck
Edit: Touched it by minimising it as well.
src/pkgcheck/checks/network.py
Outdated
| result = future.result() | ||
| if result is not None: |
There was a problem hiding this comment.
| result = future.result() | |
| if result is not None: | |
| if (result := future.result()): |
There was a problem hiding this comment.
_UrlCheck and PypiAttestationAvailableCheck do this as well. Should I touch these?
There was a problem hiding this comment.
Realised I could minimise duplicated code by using _UrlCheck. Didn't touch them but I removed this from the new check.
PypiAttestationAvailableCheck looks like it could get minimised as well.
Edit: Touched them.
694a268 to
4269370
Compare
Signed-off-by: Alfred Wingate <parona@protonmail.com>
4269370 to
9f22eac
Compare
Signed-off-by: Alfred Wingate <parona@protonmail.com>
Signed-off-by: Alfred Wingate <parona@protonmail.com>
4ca63cf to
b8f8b4b
Compare
|
The code looks fine- these are just potential points of polish or questions about being brutally paranoid about robustness ;)
I don't know that code in question, but that sounds like something our test utilities should be enhanced to support. Mind cutting a ticket laying out what you think it should be? IE, what would've allowed you to use it rather than having to sidestep it? |
|
|
||
| result = future.result() | ||
| if result is not None: | ||
| if result := future.result(): |
There was a problem hiding this comment.
Is there any scenario where result can validly be not bool(future.result()) for a returned result?
In terms of actual 'api' safety, a check of future.done() is safer, but it's also a more obnoxious chain of calls..
I mention it in reading the code; this is neither a "must" nor a "should"; just an observation since this is being converted away from is None
|
|
||
|
|
||
| class PyPIAttestationAvailableCheck(NetworkCheck): | ||
| class PyPIAttestationAvailableCheck(_UrlCheck): |
There was a problem hiding this comment.
unrelated to your change, but we should promote _UrlCheck to a public base, albeit after lacing ABC through it.
| self._schedule_check(self._provenance_check, filename, url, executor, futures, pkg=pkg) | ||
|
|
||
|
|
||
| class DetachedSignatureAvailable(results.VersionResult, results.Info): |
There was a problem hiding this comment.
Just FYI- none of this inheritance chain currently sets __slots__, but they'll be converted over. That's both to prevent accidental access, and for serialization speed reasons..
No change needed here, again, just FYI.
|
|
||
| def _verifysig_check(self, filename, url, *, pkg): | ||
| """Check for typical verify sig URLS.""" | ||
| result = None |
There was a problem hiding this comment.
Thoughts on converting this flow to just be returns? Your code is fine- there's no possible uninitialized scenario, but if you just lace returns in, the possibility goes away in full while simplifying the control flow considerations.
IE.
try:
# Need redirects to deal with the variance of file servers and urls
response = self.session.head(url, allow_redirects=True)
except RequestError:
# should this be debug logged, or does something further down the chain log it?
return None
except SSLError as e:
return SSLCertificateError("SRC_URI", url, str(e), pkg=pkg)
content_type = response.headers.get("Content-Type")
# Filtering out text/html matches is useful due to possible false matches with authentication
if (
response.ok
and content_type is not None
and not content_type.startswith("text/html")
):
return DetachedSignatureAvailable(filename, url, pkg=pkg)
return None
That's just an observation of a possible improvement.
There was a problem hiding this comment.
Actually, one tweak here simplifies things:
# Filtering out text/html matches is useful due to possible false matches with authentication
if (
response.ok
and not response.headers.get("Content-Type", "").startswith("text/html")
):
You can skip the intermediate content_type
I meant that with other network tests are simpler with having only one request triggered. So instead of simply setting requests = ... etc in responses.py, I had to have a function where I kept the implementation details for Session in mind. (Which is where Session: I don't see an issue with it as is, it currently means there is some ready made boilerplate for the next time its needed. |
@thesamesam was interested in this
I couldn't fully follow precedent with the network tests. The check deals with multiple network requests in one go so just a mocking of the returnvalue wasn't enough.